A smart vision system-on-a-chip design based on programmable neural processor integrated with active pixel sensor

A low power smart vision system based on a large format (currently 1KxlK) active pixel sensor (APS) integrated with a programmable neural processor for fast vision applications is presented. The concept of building a low power smart vision system is demonstrated by a system design, which is composed with an APS sensor, a smart image window handler, and a neural processor. The paper also shows that it is feasible to put the whole smart vision system into a single MCM chip in a standard CMOS technology. This smart vision system on-a-chip can take the combined advantages of the optics and electronics to achieve ultra-high-speed smart sensory information processing and analysis at the focal plane . The proposed system will enable many applications including robotics and machine vision, guidance and navigation, automotive applications, and consumer electronics. Future applications will also include scientific sensors such as those suitable for highly integrated imaging systems used in NASA deep space and planetary spacecraft. 1. Low Power Smart Machine Vision System Figure 1 shows a system diagram of the proposed smart vision system. The functional blocks include: (a) an active pixel sensor, (b) a smart image window handler, (c) a programmable neural processor, and (d) a host interface and timing control card. The APS is used as the optical sensing array in the system. The smart window handler manipulates the APS image data and provides the windowed image for the neural processor. The neural processor is programmed to perform various vision tasks in high speed due to its massively parallel computing structures and learning capabilities. The host computer through host interface and timing control card controls the APS sensor, the smart image window handler, and the programmable neural processor. The output image or vision science data will be displayed by the host computer. It is feasible to build the proposed smart vision system in a single CMOS chip. This smart vision system on-a-chip can take the combined advantages of the optics and electronics to achieve low-power high-speed smart sensory hformation processing and analysis at the focal plane. Tbe proposed system will enable many applications including robotics and machine vision, guidance and navigation, automotive applications, and consumer electronics. Future applications will also include scientific sensors such as those suitable for highly integrated imaging systems used in NASA deep space and planetary spacecraft. The following sections describe technical details of each building block of the proposed smart vision system and also show the feasibility to put the whole system into a single chip in a standard low power CMOS technology. 2. CMOS Active Pixel Sensor A low power CMOS APS camera-on-a-chip has been developed for producing imaging systems that can be manufactured with low cost, low power, and with excellent imaging quality [ 11. competing technology for image sensors. However, CCDs cannot be easily integrated with CMOS without additional fabrication complexity. In addition, CCDs require two-order-of-magnitude higher power dissipation than that of APS. The CCD does not have the windowing capability to provide the input data to the neural processor. On the other hand, an APS imager does not have the above limitations and it is the suitable candidate for the proposed smart vision system. The lKxlK APS is used as the optical sensing array and integrated with the neural processor to build the smart vision system for high definition vision applications. A low power lKxlK CMOS APS (operate from a +3.3 V supply) using 0.55 Om n-well process was designed and characterized at JPL. Testing results show that the large format APS with small feature size (10 micron pixel pitch) is capable of excellent imaging performance. Charge-coupled devices (CCDs) are currently the